Methodological transparency in applied linguistics and its consequences for the quality and scope of research

Methodological transparency constitutes a central tenet of the open science movement that is sweeping across many disciplines. Drawing on the burgeoning meta-science that has investigated methodological practices in applied linguistics, particularly in the area of second language learning and teaching, this chapter outlines key characteristics of methodological transparency, focusing on the reporting and availability of materials, data, coding, and analysis procedures. The chapter summarizes empirical evidence about some of the negative consequences of a lack of methodological transparency, such as how it severely weakens our capacity to understand, evaluate, and replicate research. Whilst noting a number of important challenges ahead, the chapter highlights key practices and infrastructure that are now available to researchers, institutions, funders, and editors to promote a more collaborative, sustainable, and replicable research effort

A Methodological Synthesis of Self-Paced Reading in Second Language Research : Methodological synthesis of SPR tests

Self-paced reading tests (SPRs) are being increasingly adopted by second language (L2) researchers. Using SPR with L2 populations presents specific challenges, and its use is still evolving in L2 research (as well as in first language research, in many respects). Although the topic of several narrative overviews (Keating & Jegerski, 2015; Roberts, 2016), we do not have a comprehensive picture of its usage in L2 research. Building on the growing body of systematic reviews of research practices in applied linguistics (e.g., Liu & Brown, 2015; Plonsky, 2013), we report a methodological synthesis of the rationales, study contexts, and methodological decision making in L2 SPR research. Our comprehensive search yielded 74 SPRs used in L2 research. Each instrument was coded along 121 parameters, including: reported rationales and study characteristics, indicating the scope and nature of L2 SPR research agendas; design and analysis features and reporting practices, determining instrument validity and reliability; and materials transparency, affecting reproducibility and systematicity of agendas. Our findings indicate an urgent need to standardize the use and reporting of this technique, requiring empirical investigation to inform methodological decision making. We also identify several areas (e.g., study design, sample demographics, instrument construction, data analysis, and transparency) where SPR research could be improved to enrich our understanding of L2 processing, reading, and learning

Eliciting and measuring L2 metaphoric competence : three decades on from Low (1988)

In 1988 and 2006, Applied Linguistics published Low’s and Littlemore and Low’s seminal theoretical accounts of second language (L2) metaphoric competence (MC). Meanwhile, attempts to elicit metaphor-related skills and competencies have been mixed. Instrumentation has varied in reliability, been limited in scope, and used arguably flawed reliability coefficients (McNeish, 2018). Factor analysis, used in first language (L1) MC and other areas of L2 research (e.g., Loewen et al. 2009) has not been used to explore variables. To address these issues, we developed a large battery of MC tests to elicit Low/Littlemore’s constructs, administering it to 112 L1 Mandarin speakers of L2 English and 31 L1 English speakers. Data cleaning revealed some operationalization challenges, but resulting overall reliability was high and demonstrated innovative use of ordinal omega as a powerful alternative to Cronbach’s alpha (Plonsky & Derrick, 2016). Exploratory factor analysis suggested four L2 MC variables: productive illocutionary MC, Metaphor language play, Topic/Vehicle acceptability, and Grammatical MC, broadly supporting Low/Littlemore’s proposals

IRIS : a new resource for second language research

IRIS is a digital database of materials used to collect data for second language research that went fully live in August 2012. At the time of writing (May 2013), there have been over 3,300 downloads of materials held on the site, and almost 11,000 visits, demonstrating a clear need for the resource. IRIS has the potential to make a step-change in the way our field works, impacting the nature and rate of second language (L2) research. Here we outline the motivations for the project, what the IRIS database consists of, how it is structured, and the future of the resource

A meta-analysis of sensitivity to grammatical information during self-paced reading: Towards a framework of reference for reading time effect sizes.

Despite extensive theoretical and empirical research, we do not have estimations of the magnitude of sensitivity to grammatical information during L2 online processing. This is largely due to reliance on null hypothesis significance testing (Plonsky, 2015). The current meta-analysis draws on data from one elicitation technique, self-paced reading, across 57 studies (N = 3,052), to estimate sensitivity to L2 morphosyntax and how far L1 background moderates this. Overall, we found a reliable sensitivity to L2 morphosyntax at advanced proficiencies (d =.20, 95% CIs.15,.25), with some evidence that this was reliably lower than for native speakers (NSs). These patterns were not generally moderated by linguistic feature or sentence region. However, effects for anomaly detection were larger among NSs than L2 learners and the effects among L2 learners appeared to show a trend toward L1 influence. Finding smaller effects than in other subdomains, we provide an initial framework of reference for L2 reading time effect sizes

Signatures of automaticity during practice : Explicit instruction about L1 processing routines can improve L2 grammatical processing.

This study examined the extent to which explicit instruction about first language (L1) and second language (L2) processing routines improved the accuracy, speed, and automaticity of learners' responses during sentence interpretation practice. Fifty-three English-speaking learners of L2 French were assigned to one of the following treatments: (a) a core treatment consisting of L2 explicit information (EI) with L2 interpretation practice (L2-only group); (b) the same L2 core+L1 practice with L1 EI (L2+L1 group); or (c) the same L2 core+L1 practice but without L1 EI (L2+L1prac group). Findings indicated that increasing amounts of practice led to more accurate and faster performance only for learners who received L1 EI (L2+L1 group). Coefficient of variation analyses (Segalowitz & Segalowitz, 1993) indicated knowledge restructuring early on that appeared to lead to gradual automatization over time (Solovyeva & DeKeyser, 2017; Suzuki, 2017). Our findings that EI and practice about L1 processing routines benefited the accuracy, speed, and automaticity of L2 performance have major implications for theories of L2 learning, the role of L1 EI in L2 grammar learning, and L2 pedagogy

Using explicit instruction about L1 to reduce crosslinguistic effects in L2 grammar learning : Evidence from oral production in L2 French.

This study advances previous research about the effects of explicit instruction on second language (L2) development by examining learners’ use of verbal morphology following different types of explicit information (EI) and comprehension practice. We investigated the extent to which additional EI about the first language (L1) can reduce the effects of crosslinguistic influence in L2 oral production. Sixty-nine English-speaking learners of L2 French undertook either: (a) a ‘core’ treatment of EI about the L2 with L2 comprehension practice, (b) the same L2 core + L1 comprehension practice, (c) the same L2 core + L1 comprehension practice + EI about L1, or (d) outcome tests only. Results showed that providing additional EI about the L1 benefitted the accuracy of oral production immediately after the instruction and then 6 weeks later. These results suggest that tailoring instruction, specifically the nature of the EI, to the nature of the learning problem can facilitate L2 learning. In particular, EI about L1 can facilitate L2 learning by increasing learners’ awareness of similarities and differences in how L1 and L2 express the same meanings

Teachers’ views on recognising and using home languages in predominantly monolingual primary schools.

The use of home languages has previously been advocated in highly multilingual UK classrooms (e.g. Conteh, 2007; Kenner et al., 2008; McGilp, 2014). However, drawing on the home languages and cultural insight of children who use English as an Additional Language (EAL) may also have important social and academic benefits in contexts where monolingualism is the norm. Conducted in a small local authority in England with low numbers of children who use EAL, this study investigated a) primary teachers’ views on implementing language awareness activities, using pupils who speak languages other than English as a linguistic and cultural resource, via interviews and questionnaires and b) the amount and nature of references made to home languages during classroom observations. Although the teachers did not refer to or use home languages on a day-to-day basis, they generally showed willingness to consider implementing certain activities which incorporated them. However, largely, the teachers had not previously contemplated such practice. They did not reference any academic benefits to promoting linguistic diversity but were more aware of the potential social benefits. They also lacked confidence in particular areas (e.g. linguistic knowledge) as well as showing a strong awareness of issues such as the importance of English